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ABSTRACT 

Recent  technological  advances  allow  symbology  to  be  displayed  on  the  pilot’s  visor.  A  major  benefit  of  this  is  that  the 
pilot  will  be  able  to  take  this  information  with  them  when  they  look  off-boresight.  However,  when  looking  off-boresight  the 
question  arises  as  to  what  is  the  best  orientation,  or  frame  of  reference,  for  attitude  symbology  against  the  horizon  (i.e., 
forward  or  line-of-sight)  in  order  to  maximize  interpretation  and  performance.  This  study  tested  five  different  symbologies 
(standard  HUD,  visually  coupled  acquisition  and  targeting  symbology,  arc  segmented  attitude  reference,  theta  ball,  and  non- 
distributed  flight  reference)  of  which  three  have  both  forward  and  line-of-sight  orientations.  The  experiment  consisted  of  two 
different  tasks,  with  the  pilots  performing  either  facing  the  monitor  or  rotated  90°  and  looking  over  their  shoulder  (off- 
boresight).  In  the  first  task,  pilots  maintained  straight  and  level  flight  with  simulated  turbulence.  The  second  task  had  pilots 
interpret  a  static  representation  of  their  attitude  and  respond  via  a  key  press,  and  then  the  display  went  live  and  they  had  to  fly 
to  a  new  commanded  attitude.  This  second  task  was  similar  to  a  recovery  from  unusual  attitude  methodology,  except  the  end 
state  was  never  straight  and  level.  Instead,  a  second  unknown  end  state  attitude  was  commanded  by  the  experiment.  Results 
indicate  that  performance  is  better  when  the  symbology  is  forward  as  opposed  to  line-of-sight  referenced.  Further, 
performance  was  best  in  both  tasks  for  the  non-distributed  flight  reference.  We  discuss  these  results  in  terms  of  implications 
for  helmet-mounted  display  symbology  design. 

Keywords:  helmet-mounted  display,  symbology,  attitude,  flight  reference,  off-boresight. 

1.  INTRODUCTION 

A  primary  benefit  of  helmet-mounted  display  (HMD)  symbology  is  that  it  allows  the  display  of  important  tactical 
information  for  both  air-to-air  and  air-to-ground  applications.  Moreover,  it  allows  the  pilot  to  take  this  information  off- 
boresight  thus  giving  an  advantage  in  high  off-boresight  tactical  maneuvers.  In  fact,  previous  research  has  shown  several 
benefits  of  using  HMDs,  including  the  fact  that  they  allow  pilots  to  look  further  off-boresight  and  for  longer  periods  of  time 
than  if  they  didn’t  have  an  HMD. 1,2,3 

However,  given  that,  with  HMDs,  pilots  will  spend  more  time  looking  off-boresight,  the  need  arises  to  not  only  give 
targeting  information  but  also  ownship  status  information  (airspeed,  heading,  altitude,  and  attitude)  so  as  to  maximize  the 
situation  awareness  (SA)  of  the  pilot.  Previous  research  has  shown  that  in  laboratory  experiments  as  well  as  flight  tests,  pilots 
concur  that  ownship  status  information  should  be  included  off-boresight. . . 

There  is,  however,  a  question  that  arises  when  ownship  status  information  is  taken  off-boresight.  Namely,  what  is  the  best 
orientation,  or  frame  of  reference,  for  attitude  symbology  against  the  horizon  (i.e.,  forward  versus  line-of-sight)  in  order  to 
maximize  interpretation  and  performance.  To  investigate  this  question  two  different  tasks  were  tested  (attitude  maintenance 
and  commanded  attitude)  facing  the  monitor  (on-boresight)  and  turned  ninety  degrees  so  that  the  pilot  had  to  look  to  the  right 
to  see  the  monitor  (off-boresight).  To  explain  the  frame  of  reference  manipulation,  consider  the  case  where  one  puts  the 
standard  head-up  display  (HUD)  symbology  on  an  HMD.  If  the  pilot  looks  off-boresight,  and  the  attitude  symbology  (i.e., 
bank  and  climb/dive  angle)  looks  the  same  as  if  it  were  still  on  the  HUD,  this  would  be  termed  forward  referenced.  In  this 
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case,  looking  ninety  degrees  off-boresight  the  symbology  would  no  longer  be  conformal  with  the  horizon.  However,  if  the 
attitude  component  became  conformal  with  the  horizon,  this  would  be  termed  the  line-of-sight  frame  of  reference.  The 
purpose  of  this  study  was  to  evaluate  a  number  of  candidate  symbologies  as  well  as  the  two  frame  of  reference  manipulations 
to  see  which  gave  the  best  ownship  status  information  during  the  two  different  tasks. 

2.  METHOD 

2.1  Participants 

A  total  of  10  pilots  volunteered  to  participate  in  the  study  with  between  41  and  3000  hours  of  flight  experience. 

2.2  Symbol  sets 

There  were  five  symbology  sets  compared  in  the  present  study.  Each  pilot  participated  in  the  experiment  with  only  one  of 
the  five  symbol  sets.  Three  of  the  five  symbol  sets  had  both  a  forward  as  well  as  a  line-of-sight  version:  standard  HUD,  Theta 
Ball,  and  the  Arc-Segmented  Attitude  Reference  /  Wing  Arc  Vector  (ASARAVAV).  The  other  two  symbol  sets  were  forward 
referenced  only:  the  Visually  Coupled  Acquisition  and  Targeting  System  (VCATS),  and  the  Non-Distributed  Flight  Reference 
(NDFR).  Throughout  the  experiment,  each  symbol  set  displayed  only  airspeed,  heading,  altitude,  and  attitude  (bank  and 
climb/dive  angle)  information. 

2.2.1  Standard  HUD  symbology 

A  representational  version  of  MTL-STD-1787B  8  was  used  and  can  be  seen  in  Figure  1.  In  this  figure,  and  the  figures  to 
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Figure  1 .  Standard  HUD  symbol  set. 

follow,  an  ownship  attitude  symbol  is  shown  (approximately  28  degrees  nose  high)  followed  by  the  representative  symbol  set. 
Further,  if  the  symbol  set  has  a  line-of-sight  version  it,  too,  will  be  shown.  Note  in  Figure  I  several  things  about  the  line-of- 
sight  version  of  the  HUD.  First,  there  is  a  wing  reference  to  indicate  that  the  symbology  represents  line-of-sight,  and  therefore 
conformal,  information.  Second,  since  the  mechanization  is  conformal  with  the  horizon,  the  pitch  ladder  now  represents  roll, 
and  likewise  the  amount  of  pitch  is  read  via  the  horizon  line  and  the  wing  reference. 

2.2.2  Theta  Ball  symbology 

A  second  symbology  called  Theta  Ball  9,10  was  used  and  can  be  seen  in  Figure  2.  Theta  Ball  was  developed  as  a 
symbology  that  integrates  a  standard  attitude/direction  indicator  (ADI)  and  standard  HUD.  Included  within  the  ball  are  an 
enhanced  horizon  line  as  well  as  alphanumerical  cardinal  headings.  There  are  also  longitudinal  lines  that  increment  in  45 
degree  sections,  and  the  ball  rotates  about  all  three  axes  thus  representing  roll,  climb/dive,  and  heading.  Further,  the  top  half 
of  the  ball  has  solid  lines  while  the  bottom  half  is  segmented.  This  symbology  also  has  both  a  forward  and  line-of  sight 
version  as  can  be  seen  in  Figure  2. 
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Figure  2.  Theta  Ball  symbol  set. 
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Figure  3.  ASARAVAV  symbol  set. 


2.2.3  Arc-Segmented  Attitude  Reference  /  Wing  Arc  Vector  (ASARAVAV)  symbology 

A  third  symbology  called  ASARAVAV  11  is  shown  in  Figure  3.  The  ASAR,  sometimes  referred  to  as  an  “orange  peel”, 
was  developed  for  the  German  Air  Force  and  represents  attitude  by  an  arc  surrounding  a  fixed  climb/dive  symbol.  During 
straight  and  level  flight  the  arc  is  a  half  circle,  and  as  climb/dive  increases  (as  in  Figure  3)  the  arc  area  shrinks.  Likewise,  as 
climb/dive  decreases  the  arc  increases  in  area  to  form  a  complete  circle  at  90  degrees  nose  high.  When  performing  a  roll 
maneuver  the  arc  will  rotate  about  the  fixed  climb/dive  symbol.  This  symbol  set  is  the  last  one  in  the  experiment  that  has  a 
line-of-sight  version  as  can  be  seen  in  Figure  3. 

2.2.4  Non  Distributed  Flight  Reference  (NDFR)  symbology 

NDFR  3,12  was  the  fourth  symbol  set  used  and  is  shown  in  Figure  4.  This  is  the  first  of  the  two  symbol  sets  that  did  not 
have  a  line-of-sight  version.  Note  that  the  reason  for  this  is  there  is  no  room  for  the  wing  reference  in  the  NDFR  symbol  set. 
NDFR  was  developed  by  the  Air  Force  Research  Lab  as  a  flight  reference  that  takes  up  less  real  estate  than  other 
contemporary  symbol  sets.  The  mechanization  of  NDFR  is  similar  to  ASARAVAV  in  that  both  use  the  “orange  peel”  arc  for 
attitude  information.  However,  NDFR  brings  in  airspeed,  heading,  and  altitude  around  a  flight  path  marker  to  give  the  pilot  all 
of  the  ownship  status  information  in  a  small  well  defined  area.  Previous  research  with  NDFR  has  shown  that  pilots  prefer  this 
compressed  information  and  often  refer  to  it  as  a  quick  information  stamp  that  allows  them  to  help  maintain  their  situation 
awareness. 3 

2.2.5  Visually-Coupled  Acquisition  and  Targeting  System  (VCATS)  symbology 

The  final  symbol  set  used  was  VCATS  and  is  shown  in  Figure  5.  This  symbol  set  was  developed  by  the  USAF  and  the 
Boeing  Corporation  as  a  high  altitude  ownship  attitude  reference.  Attitude  is  mechanized  by  a  single  split  horizon  with  a 
climb/dive  angle  reference.  The  horizon  line  will,  during  a  roll  maneuver,  rotate  about  the  fixed  climb/dive  symbol.  In  a 
climb,  the  line  will  move  above  the  climb/dive  symbol.  In  a  dive,  the  line  drops  below  the  symbol  and  becomes  segmented. 
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Figure  4.  Non  Distributed  Flight  Reference  symbol  set. 
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Figure  5.  Visually-Coupled  Acquisition  and  Targeting  System  symbol  set. 


2.3  Experimental  tasks 

The  entire  experiment  consisted  of  four  parts:  free  flight,  criterion,  attitude  maintenance  task,  and  commanded  attitude 
task.  Pilots  were  randomly  assigned  to  one  of  the  five  candidate  symbol  sets  upon  arrival.  They  were  briefed  as  to  the  purpose 
of  the  experiment,  and  given  a  verbal  as  well  as  pictorial  overview  of  the  symbol  set  they  would  be  using.  All  stimuli  were 
presented  on  a  Silicon  Graphics  02  with  a  17-inch  monitor  (1024  x  768  resolution). 

2.3.1  Free  flight 

The  first  part  of  the  experiment  allowed  the  pilots  to  familiarize  themselves  with  their  selected  symbol  set,  for  as  long  as 
they  needed  to  feel  they  understood  the  mechanizations  of  the  symbology.  Throughout  the  experiment  there  was  always  a  grid 
representing  ground  and  a  dark  gray  sky.  Initially,  during  free  flight,  pilots  faced  the  monitor.  However,  as  part  of  each  task 
would  require  them  to  fly  “looking  off-boresight”,  when  they  felt  comfortable  with  the  flight  dynamics  and  the  symbology, 
they  were  turned  90  degrees  and  allowed  to  free-fly  looking  off-boresight.  It  should  be  noted  here  that  the  experimenter  then 
pressed  a  button  that  changed  the  dynamics  of  the  simulation  such  that  the  outside  view  shown  on  the  monitor  looked  as  it 
would  if  flying  while  looking  off-boresight.  All  pilots  did  this  initially  using  the  forward  referenced  version  of  the  symbol  set, 
but  in  the  case  where  a  pilot's  symbology  had  a  line-of-sight  reference,  they  were  allowed,  once  they  felt  comfortable  with  the 
forward  referenced  symbology,  to  fly  with  the  line-of-sight  referenced  version  of  the  symbology. 

2.3.2  Criterion  task 

Once  the  pilot  felt  comfortable  with  the  mechanization  of  the  symbology,  both  on-  and  off-boresight,  the  criterion  phase 
of  the  experiment  began.  In  this  phase  the  purpose  was  to  get  each  pilot  used  to  flying  straight  and  level  without  any 
symbology  with  simulated  turbulence.  The  rationale  was  to  make  sure  each  pilot  was  proficient  with  the  flight  dynamics  to 
continue  the  experiment.  The  turbulence  was  designed  so  that  it  was  replicable  but  not  predictable.  The  disturbance  function 
that  simulated  the  turbulence  was  a  gaussian  random  function  that  was  run  at  60  Hz.  The  function  was  scaled  such  that  if  no 


stick  input  was  given,  the  root  mean  squared  (RMS)  error  would  be  8  (note  that  the  unit  is  arbitrary  as  it  is  a  function  of  the 
disturbance  function)  for  both  roll  and  climb/dive.  Thus  if  one  could  perfectly  predict  and  dampen  the  turbulence,  a  RMS 
error  score  of  0  would  be  obtained  for  both  roll  and  climb/dive.  Further,  with  no  stick  input  the  attitude  deviations  would  be 
no  more  than  +/-  60  degrees  for  roll  and  +/-  30  degrees  for  climb/dive.  The  pilot  began  each  trial  by  pressing  the  trigger  on 
the  stick,  which  initiated  the  trial,  beginning  with  3  seconds  of  no  turbulence,  then  17  seconds  of  the  simulated  turbulence,  and 
ending  with  7  seconds  of  no  turbulence.  During  the  trial,  pilots  were  instructed  to  maintain  straight  and  level  flight.  They  were 
further  instructed  that,  once  the  turbulence  ended,  they  were  to  make  sure  they  were  back  at  straight  and  level.  After  each  trial, 
they  were  shown  the  RMS  error  values  for  both  roll  and  climb/dive  for  the  trial,  and  starting  with  the  tenth  trial  they  were 
shown  a  moving  average  of  the  last  10  trials*  RMS  error  values  for  roll  and  climb/dive.  Their  task  was  to  achieve  an  average 
RMS  error  value  of  2.9  or  below  for  roll,  and  3.4  or  below  for  climb/dive  for  ten  trials.  Once  this  was  accomplished,  the 
criterion  phase  of  the  experiment  was  completed.  All  ten  pilots  completed  criterion,  this  required  anywhere  from  10  trials  to 
80  trials. 

2.3.3  Attitude  maintenance  task 

After  reaching  criterion,  without  the  use  of  symbology,  the  attitude  maintenance  task  began.  This  task  was  similar  to  the 
criterion  phase  of  the  experiment,  but  now  the  pilot*  s  were  given  their  candidate  symbology  to  help  them  maintain  straight  and 
level  flight.  All  pilots  first  ran  56  trials  facing  the  monitor,  after  which  they  were  turned  90  degrees  and  ran  56  more  trials 
looking  off-boresight.  However,  if  a  pilot’s  candidate  symbology  set  had  both  forward  and  line-of-sight  versions,  they  ran  56 
forward  and  56  line-of-sight  trials.  The  forward  versus  line-of-sight  manipulation  was  counterbalanced  such  that  every  other 
pilot  ran  the  line-of-sight  trials  before  running  the  forward  referenced  trials.  As  before,  after  each  trial,  RMS  error  for  roll  and 
climb/dive  were  given  as  well  as  a  moving  average  RMS  error  of  ten  trials.  Pilots  were  not  told  to  reach  a  criterion  but  to  do 
the  best  they  could  and  to  use  the  RMS  error  as  a  gauge  as  to  their  performance. 

2.3.4  Commanded  attitude  task 

The  second  part  of  the  experiment  was  the  commanded  attitude  task  in  which  pilots  had  to  interpret  the  symbology  given 
to  them  and  then  fly  to  a  new  commanded  attitude.  Figure  6  shows  a  screen  shot  of  the  beginning  of  a  trial  using  standard 
HUD.  The  first  task  of  the  pilot  (the  interpretation  task)  was  to  pick  which  of  the  eight  command  attitude  cues  corresponded 
to  the  attitude  of  the  symbology  (i.e.,  in  this  example  45  degrees  banked  left  or  number  4).  The  command  attitude  cues  were, 
starting  on  the  top  row:  nose  high,  nose  low,  roll  right,  and  roll  left;  on  the  bottom  row:  inverted  nose  high,  inverted  nose  low, 
inverted  roll  left,  and  inverted  roll  right.  The  pilots  were  told  to  select  the  correct  command  attitude  cue  via  a  bank  of  eight 


Figure  6.  Part  one  of  the  commanded  attitude  task. 


buttons  as  quickly  as  possible.  If  incorrect,  the  words  “Incorrect  Try  Again”  appeared  and  the  pilot  was  allowed  further 
selections  until  the  correct  answer  was  given. 

Once  the  correct  answer  was  given,  the  display  would  “go  live”  (commanded  attitude  task)  and  the  pilot  was  given  a 
command  attitude  cue  to  fly  to  a  new  attitude.  This  cue  was  one  of  the  previously  displayed  choices  for  the  first  part  of  the 
task  minus  the  attitude  that  the  pilot  had  just  correctly  identified.  To  end  the  trial  the  pilot  had  to  maintain  the  specified  new 
attitude  for  approximately  two  seconds.  Pilots  were  told  before  starting  the  second  part  of  the  experiment  that  the  nose  high 
and  nose  low  attitudes  were  28  degrees  and  the  rolls  were  45  degrees.  As  in  the  first  part  of  the  experiment,  56  trials  were 
completed  facing  the  monitor,  followed  by  either  56  or  128  trials  looking  off-boresight  (56  forward  and  56  line-of-sight) 
depending  upon  the  pilot’s  candidate  symbology.  It  should  be  noted  that  this  phase  of  the  experiment  was  designed  to  be  more 
of  an  operational  task  than  a  typical  recovery  task  and  thus  further  assess  the  usefulness  of  the  symbol  sets  being  tested. 

3.  RESULTS 


3.1  Analysis  of  the  attitude  maintenance  task 

The  analysis  of  the  attitude  maintenance  task  was  performed  as  a  between-subjects  design  with  two  pilots  per  symbol  set. 
For  both  the  forward  and  off-boresight  conditions,  as  well  as  the  frame  of  reference  conditions  (forward  versus  line-of-sight) 
the  RMS  error  for  roll  and  climb/dive  were  analyzed.  To  help  normalize  the  data,  all  data  were  log  transformed  before 
analysis. 

3.1.1  Attitude  maintenance  task  -  Forward  condition 

An  Analysis  of  Variance  (ANOVA)  was  performed  on  the  log  RMS  error  data  and  revealed  a  significant  effect  of  symbol 
set  for  both  climb/dive  error  F(4,280)  =  16.88,  p  <  .0001,  and  roll  F(4,280)  =  28.28,  p  <  .0001.  Post  hoc  analysis  on  these 
data  using  Tukey’s  HSD  test  at  the  .05  alpha  level  revealed  significant  differences  for  climb/dive  error  between  NDFR  and 
the  other  four  symbol  sets  (i.e.,  ASAR/WAV,  VCATS,  HUD,  and  Theta  were  not  significantly  different  from  each  other)  as 
can  be  seen  in  Figure  7.  Post  hoc  analysis  on  the  log  RMS  roll  error  revealed  no  significant  difference  between  Theta  and 
VCATS,  or  between  HUD  and  ASAR/WAV;  however  both  of  these  groupings  were  significantly  different  from  each  other 
and  from  NDFR.  These  results  are  shown  in  Figure  8. 
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Figure  7.  Log  RMS  error  for  climb/dive. 
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Figure  8.  Log  RMS  error  for  roll. 

3.1.2  Attitude  maintenance  task  -  Off-boresight  condition 

In  order  to  evaluate  the  effect  of  the  two  frame  of  reference  manipulations  (i.e.,  forward  versus  line-of-sight),  an  ANOVA 
was  performed  on  the  three  symbol  sets  that  had  both  references  (ASAR/WAV,  HUD,  and  Theta).  The  analysis  revealed  a 
significant  effect  of  frame  of  reference  for  log  RMS  climb/dive  error  F(l,666)  =  31.56,  p  <  .0001  with  the  average  forward 
referenced  errors  lower  than  the  average  line-of-sight  errors.  Analysis  of  the  log  RMS  roll  error  was  not  significant  F( 1,666)  = 
1.82,  p  >  .05.  Since  these  results  show  that  forward  referenced  symbology  was  better  than,  or  equal  to,  line-of-sight 
referenced  symbology,  a  second  ANOVA  was  performed  on  the  error  data  for  only  the  forward  referenced  conditions.  This 
analysis  revealed  a  significant  effect  of  symbol  set  for  both  climb/dive  error  F(4,280)  =  15.02,  p  <  .0001,  and  roll  error 
F(4,280)  =  11.17,  p  <  .0001.  Post  hoc  analysis  on  these  data  using  Tukey’s  HSD  test  at  the  .05  alpha  level  revealed  no 
significant  differences  for  climb/dive  error  between  the  following  groups:  (VCATS  and  ASAR/WAV),  (ASAR/WAV,  NDFR, 
and  Theta)  and  (NDFR,  Theta,  and  HUD),  and  can  be  seen  in  Figure  9.  Post  hoc  analysis  of  the  roll  error  revealed  significant 
differences  between  VCATS  and  the  other  four  symbol  sets  (i.e.,  ASAR/WAV,  HUD,  Theta,  and  NDFR).  These  results  are 
shown  in  Figure  10. 
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Figure  9.  Log  RMS  error  for  climb/dive  forward  referenced  only. 
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Figure  10.  Log  RMS  error  for  roll  forward  referenced  only. 

3.2  Analysis  of  the  commanded  attitude  task 

The  analysis  of  the  commanded  attitude  tasks  was  performed  as  a  between-subjects  design  with  two  pilots  per  symbol  set. 
For  both  the  forward  and  off-boresight  conditions,  as  well  as  the  frame  of  reference  conditions  (forward  versus  line-of-sight), 
the  reaction  times  (RTs)  for  the  interpretation  task  were  analyzed,  and  for  the  commanded  attitude  task  the  initial  stick  input 
RT  and  total  RT  (i.e.,  total  time  to  fly  to  new  commanded  attitude)  were  analyzed.  To  help  normalize  the  data  all  data  was  log 
transformed  before  analysis  as  was  done  in  the  attitude  maintenance  task. 

3.2.1  Commanded  attitude  tasks  -  Forward  condition 

An  ANOVA  was  performed  on  the  interpretation  task  log  RT  data  and  revealed  a  significant  effect  of  symbol  set 
F(4,480)  =  4.21,  p  <  .0023.  Post  hoc  analysis  on  these  data  using  Tukey’s  HSD  test  at  the  .05  alpha  level  revealed  significant 
differences  only  for  Theta  and  NDFR.  There  were  no  significant  differences  between  Theta,  VCATS,  ASAR/WAV  and  HUD, 
or  between  VCATS,  ASAR/WAV,  HUD,  and  NDFR.  These  results  are  shown  below  in  Figure  11.  For  the  commanded 
attitude  task,  an  ANOVA  was  performed  on  the  initial  stick  input  log  RT  data  and  revealed  a  significant  effect  of  symbol  set 
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Figure  11.  Log  (RT)  for  interpretation  task. 

F(4,80)  =  24.39,  p  <  .0001.  Post  hoc  analysis  on  these  data  using  Tukey’s  HSD  test  at  the  .05  alpha  level  revealed  no 
significant  differences  between  Theta  and  HUD,  or  between  ASAR  and  VCATS.  However  both  of  these  groupings  were 
significantly  different  from  each  other,  and  from  NDFR,  and  these  results  can  be  seen  in  Figure  12.  Finally,  an  ANOVA  was 
performed  on  the  total  log  RT  data  and  revealed  a  significant  effect  of  symbol  set  F(4,278)  =  10.90,  p  <  .0001.  Post  hoc 
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Figure  12.  Log  (Initial  stick  input  RT)  for  commanded  attitude  task. 

analysis  on  these  data  using  Tukey’s  HSD  test  at  the  .05  alpha  level  revealed  no  significant  differences  between  HUD, 
VCATS,  and  ASAR/WAV,  or  between  VCATS,  ASAR/WAV,  and  NDFR,  however  Theta  was  significantly  different  from  all 
of  the  symbol  sets  as  can  be  seen  in  Figure  13. 

3.2.2  Commanded  attitude  tasks  -  Off-boresight  condition 

As  in  the  attitude  maintenance  analysis,  to  evaluate  the  effect  of  the  two  frame  of  reference  manipulations,  an  ANOVA 
was  performed  on  the  three  symbol  sets  that  had  both  references  (ASAR/WAV,  HUD,  and  Theta).  The  analysis  was  first 
conducted  on  the  interpretation  log  RT  data  revealed  no  significant  effect  of  frame  of  reference  for  RT  F( 1,590)  =  1.60,  p  > 
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Figure  13.  Log  (Total  RT)  for  commanded  attitude  task. 

.05.  Since  these  results  show  that  forward  referenced  symbology  was  equal  to  line-of-sight  referenced  symbology,  a  second 
ANOVA  was  performed  on  the  interpretation  task  log  RT  data  for  only  the  forward  referenced  conditions.  This  analysis 
revealed  a  significant  effect  of  symbol  set  F(4,499)  =  15.95,  p  <  .0001.  Post  hoc  analysis  on  these  data  using  Tukey’s  HSD 
test  at  the  .05  alpha  level  revealed  no  significant  differences  between  VCATS,  HUD,  Theta,  and  ASAR/WAV,  but  all  four 
were  significantly  different  from  NDFR  as  can  be  seen  in  Figure  14. 
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Figure  14.  Log  (RT)  for  interpretation  task  forward  referenced  only. 

For  the  commanded  attitude  task,  an  ANOVA  was  performed  on  the  three  symbol  sets  that  had  both  references 
(ASAR/WAV,  HUD,  and  Theta)  to  check  the  frame  of  reference  manipulation.  The  analysis  was  conducted  on  both  the  initial 
stick  input  log  RT  data  as  well  as  the  log  Total  RT  data.  The  analysis  on  the  initial  stick  data  revealed  a  significant  effect  of 
frame  of  reference  F(l,379)  =  7.19,  p  <  .001  with  forward  referenced  having  lower  RTs  than  line-of-sight  referenced 
symbology.  The  total  RT  data  however  revealed  no  significant  difference  for  frame  of  reference  F(l,662)  =  2.4,  p  >  .05.  As 
before,  since  the  analysis  revealed  that  forward  referenced  was  either  better  than,  or  the  same  as,  line-of-sight  referenced 
symbology,  another  analysis  was  conducted  on  the  forward  referenced  only  conditions.  This  analysis  revealed  a  significant 
effect  of  symbol  set  for  initial  stick  input  RT  F(4,335)  =  4.77,  p  <  .0009,  as  well  as  total  RT  F(4,554)  =  12.51,  p  <  .0001.  Post 
hoc  analysis  on  these  data  using  Tukey’s  HSD  test  at  the  .05  alpha  level  revealed  no  significant  differences  for  initial  stick 
input  RT  between,  HUD,  ASAR/WAV,  NDFR,  and  Theta,  or  between  ASAR/WAV,  NDFR,  Theta,  and  VCATS,  and  can  be 
seen  in  Figure  15.  Post  hoc  analysis  the  total  RT  data  revealed  significant  differences  between  NDFR  and  the  other  four 
symbol  sets  (i.e.,  ASAR/WAV,  HUD,  Theta,  and  VCATS).  These  results  are  shown  in  Figure  16. 
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Figure  15.  Log  (initial  stick  input  RT)  forward  referenced  only. 
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Figure  16.  Log  (total  RT)  forward  referenced  only. 

3.  DISCUSSION 

The  purpose  of  this  experiment  was  twofold.  First,  we  wished  to  investigate  the  effect  of  two  different  frames  of  reference 
(forward  versus  line-of-sight)  for  attitude  symbology  to  be  used  both  on-  and  off-boresight.  Secondly,  we  wanted  to 
understand  what  is  the  best  symbol  set  to  use  to  present  ownship  status  information  for  two  different  aircraft  control  tasks 
(attitude  maintenance  and  commanded  attitude).  In  the  first  part  of  the  experiment,  pilots  had  to  maintain  straight  and  level 
flight  while  flying  looking  either  on  or  off-boresight.  The  results  indicate  that  when  flying  on-boresight,  the  NDFR  symbology 
allows  better  control,  as  shown  by  lower  RMS  errors  for  roll  and  climb/dive.  When  this  task  was  performed  while  looking  off- 
boresight,  the  pilots  performed  better  with  forward  as  opposed  to  line-of-sight  symbology.  Further  analysis  revealed  that,  for 
the  forward  referenced  versions  of  the  symbol  sets,  there  is  no  clear  “winner”.  However,  this  task  was  very  difficult  and  it  was 
pointed  out  by  several  of  the  pilots  that,  given  the  same  situation  in  actual  flight,  they  would  immediately  come  back  inside 
the  cockpit  and  fly  on-boresight. 

In  the  second  half  of  the  experiment,  pilots  first  had  to  interpret  the  attitude  of  the  displayed  symbology  and  then  they 
were  commanded  to  fly  to  a  new  attitude.  Analysis  indicates  that,  when  performing  the  task  on-boresight,  reaction  times  for 
choosing  the  correct  attitude  were  fastest  using  NDFR;  however  NDFR  was  not  significantly  different  from  any  of  the  other 
symbol  sets  except  Theta.  For  the  recovery  task,  initial  stick  input  was  significantly  faster  for  NDFR,  yet  total  RT  is  the  same 
for  all  symbol  sets  except  for  Theta  (the  slowest).  Further  analysis  reveals  that,  when  the  task  was  performed  looking  off- 
boresight,  performance  was  better  with  forward  referenced  symbology,  as  it  was  in  the  first  part  of  the  experiment,  for  both 
the  interpretation  task  and  the  commanded  attitude  task.  Moreover,  results  indicated  that  for  the  interpretation  task 
performance  was  best  with  NDFR.  Further,  for  the  commanded  attitude  task,  there  was  no  clear  winner  for  best  performance 
for  initial  stick  input,  however  NDFR  outperformed  the  other  symbologies  for  total  RT . 

In  summary,  we  find  that,  when  symbology  is  to  be  taken  off-boresight,  it  is  best  to  leave  it  forward  referenced.  This 
result  alleviates  the  concern  over  how  to  mechanize  a  line-of-sight  symbology.  In  other  words:  when,  as  pilot’s  head  turns, 
would  it  change  from  forward  to  line-of-sight  referenced?  Performance  results  indicate  that  pilots  perform  the  same,  if  not 
better,  during  an  off-boresight  task  when  the  symbology  is  forward  referenced.  Further,  we  also  have  shown  a  benefit  of  the 
NDFR  symbology  as  a  simple,  small  footprint,  ownship  status  symbology  that,  in  general,  performs  as  well  as,  or  in  many 
cases,  outperforms  other  symbol  sets. 
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